Neural Dynamic Focused Topic Model

Topic models and all their variants analyse text by learning meaningful representations through word co-occurrences. As pointed out by Williamson et al. (2010), such models implicitly assume that the probability of a topic to be active and its proportion within each document are positively correlated. This correlation can be strongly detrimental in the case of documents created over time, simply because recent documents are likely better described by new and hence rare topics. In this work we leverage recent advances in neural variational inference and present an alternative neural approach to the dynamic Focused Topic Model. Indeed, we develop a neural model for topic evolution which exploits sequences of Bernoulli random variables in order to track the appearances of topics, thereby decoupling their activities from their proportions. We evaluate our model on three different datasets (the UN general debates, the collection of NeurIPS papers, and the ACL Anthology dataset) and show that it (i) outperforms state-of-the-art topic models in generalization tasks and (ii) performs comparably to them on prediction tasks, while employing roughly the same number of parameters, and converging about two times faster. Source code to reproduce our experiments is available online.Comment: Accepted at Association for the Advancement of Artificial Intelligence (AAAI2023

Knowledge Augmented Machine Learning with Applications in Autonomous Driving: A Survey

The existence of representative datasets is a prerequisite of many successful artificial intelligence and machine learning models. However, the subsequent application of these models often involves scenarios that are inadequately represented in the data used for training. The reasons for this are manifold and range from time and cost constraints to ethical considerations. As a consequence, the reliable use of these models, especially in safety-critical applications, is a huge challenge. Leveraging additional, already existing sources of knowledge is key to overcome the limitations of purely data-driven approaches, and eventually to increase the generalization capability of these models. Furthermore, predictions that conform with knowledge are crucial for making trustworthy and safe decisions even in underrepresented scenarios. This work provides an overview of existing techniques and methods in the literature that combine data-based models with existing knowledge. The identified approaches are structured according to the categories integration, extraction and conformity. Special attention is given to applications in the field of autonomous driving

Studying Dynamics of User Behavior. Heterocedastic Time Series Forecasting and Clustering of Inhomogeneous Poisson Process

Complex time series patterns are generated by the behavior of a large number of different users in so the called question and answering web platforms. This calls for flexible, accurate and descriptive techniques for studying the dynamics of such systems. In this study, we extend the Sparse Input Gaussian Process formalism, in order to incorporate functional description of the input dependent noise. Such procedure also provides a regularization method that improves the accuracy of the predictions. We compare our results with the results of the other Gaussian Process methods, and apply the methodology to time series from the questions and answer web site Stackoverflow. For finding the common behavior between the users we propose the scale invariant Dynamic Piecewise Similarity measures an d the K-PSC clustering algorithm for clustering time series in order to provide much more descriptive cluster centroids then the centroids from the K-Means clustering algorithm

Introduction to machine learning with robots and playful learning

Inspired by explanations of machine learning concepts in children’s books, we developed an approach to introduce supervised, unsupervised, and reinforcement learning using a block-based programming language in combination with the benefits of educational robotics. Instead of using blocks as high-end APIs to access AI cloud services or to reproduce the machine learning algorithms, we use them as a means to put the student “in the algorithm’s shoes.” We adapt the training of neural networks, Q-learning, and k-means algorithms to a design and format suitable for children and equip the students with hands-on tools for playful experimentation. The children learn about direct supervision by modifying the weights in the neural networks and immediately observing the effects on the simulated robot. Following the ideas of constructionism, they experience how the algorithms and underlying machine learning concepts work in practice. We conducted and evaluated this approach with students in primary, middle, and high school. All the age groups perceived the topics to be very easy to moderately hard to grasp. Younger students experienced direct supervision as challenging, whereas they found Q-learning and k-means algorithms much more accessible. Most high-school students could cope with all the topics without particular difficulties

Inverse Dynamical Inheritance in Stack Exchange Taxonomies

Question Answering websites are popular repositories of expert knowledge and cover areas as diverse as linguistics, computer science, or mathematics. Knowledge is commonly organized via user defined tags which implicitly create population folksonomies. However, the interplay between latent knowledge structures and the answering behavior of users has not been fully explored yet. Here, we propose a model of a dynamical tagging process guided by taxonomies, devise a robust algorithm that allow us to uncover hidden topic hierarchies, apply our method to analyze several Stack Exchange websites. Our results show that the dynamics of the system strongly correlate with uncovered taxonomies

Inverse dynamical inheritance in stack exchange taxonomies

Question Answering websites are popular repositories of expert knowledge and cover areas as diverse as linguistics, computer science, or mathematics. Knowledge is commonly organized via user defined tags which implicitly create population folksonomies. However, the interplay between latent knowledge structures and the answering behavior of users has not been fully explored yet. Here, we propose a model of a dynamical tagging process guided by taxonomies, devise a robust algorithm that allow us to uncover hidden topic hierarchies, apply our method to analyze several Stack Exchange websites. Our results show that the dynamics of the system strongly correlate with uncovered taxonomies